Active learning for reducing labeling effort in text classification tasks

Labeling data can be an expensive task as it is usually performed manually by domain experts. This is cumbersome for deep learning, as it is dependent on large labeled datasets. Active learning (AL) is a paradigm that aims to reduce labeling effort by only using the data which the used model deems most informative. Little research has been done on AL in a text classification setting and next to none has involved the more recent, state-of-the-art Natural Language Processing (NLP) models. Here, we present an empirical study that compares different uncertainty-based algorithms with BERT$_{base}$ as the used classifier. We evaluate the algorithms on two NLP classification datasets: Stanford Sentiment Treebank and KvK-Frontpages. Additionally, we explore heuristics that aim to solve presupposed problems of uncertainty-based AL; namely, that it is unscalable and that it is prone to selecting outliers. Furthermore, we explore the influence of the query-pool size on the performance of AL. Whereas it was found that the proposed heuristics for AL did not improve performance of AL; our results show that using uncertainty-based AL with BERT$_{base}$ outperforms random sampling of data. This difference in performance can decrease as the query-pool size gets larger.Comment: Accepted as a conference paper at the joint 33rd Benelux Conference on Artificial Intelligence and the 30th Belgian Dutch Conference on Machine Learning (BNAIC/BENELEARN 2021). This camera-ready version submitted to BNAIC/BENELEARN, adds several improvements including a more thorough discussion of related work plus an extended discussion section. 28 pages including references and appendice

Active Learning for Reducing Labeling Effort in Text Classification Tasks

Labeling data can be an expensive task as it is usually performed manually by domain experts. This is cumbersome for deep learning, as it is dependent on large labeled datasets. Active learning (AL) is a paradigm that aims to reduce labeling effort by only using the data which the used model deems most informative. Little research has been done on AL in a text classification setting and next to none has involved the more recent, state-of-the-art Natural Language Processing (NLP) models. Here, we present an empirical study that compares different uncertainty-based algorithms with BERT$_{base}$ as the used classifier. We evaluate the algorithms on two NLP classification datasets: Stanford Sentiment Treebank and KvK-Frontpages. Additionally, we explore heuristics that aim to solve presupposed problems of uncertainty-based AL; namely, that it is unscalable and that it is prone to selecting outliers. Furthermore, we explore the influence of the query-pool size on the performance of AL. Whereas it was found that the proposed heuristics for AL did not improve performance of AL; our results show that using uncertainty-based AL with BERT$_{base}$ outperforms random sampling of data. This difference in performance can decrease as the query-pool size gets larger.Comment: Accepted as a conference paper at the joint 33rd Benelux Conference on Artificial Intelligence and the 30th Belgian Dutch Conference on Machine Learning (BNAIC/BENELEARN 2021). This camera-ready version submitted to BNAIC/BENELEARN, adds several improvements including a more thorough discussion of related work plus an extended discussion section. 28 pages including references and appendice

The OCareCloudS project: toward organizing care through trusted cloud services

The increasing elderly population and the shift from acute to chronic illness makes it difficult to care for people in hospitals and rest homes. Moreover, elderly people, if given a choice, want to stay at home as long as possible. In this article, the methodologies to develop a cloud-based semantic system, offering valuable information and knowledge-based services, are presented. The information and services are related to the different personal living hemispheres of the patient, namely the daily care-related needs, the social needs and the daily life assistance. Ontologies are used to facilitate the integration, analysis, aggregation and efficient use of all the available data in the cloud. By using an interdisciplinary research approach, where user researchers, (ontology) engineers, researchers and domain stakeholders are at the forefront, a platform can be developed of great added value for the patients that want to grow old in their own home and for their caregivers

Active Learning for Reducing Labeling Effort in Text Classification Tasks

Labeling data can be an expensive task as it is usually performed manually by domain experts. This is cumbersome for deep learning, as it is dependent on large labeled datasets. Active learning (AL) is a paradigm that aims to reduce labeling effort by only using the data which the used model deems most informative. Little research has been done on AL in a text classification setting and next to none has involved the more recent, state-of-the-art Natural Language Processing (NLP) models. Here, we present an empirical study that compares different uncertainty-based algorithms with BERT base as the used classifier. We evaluate the algorithms on two NLP classification datasets: Stanford Sentiment Treebank and KvK-Frontpages. Additionally, we explore heuristics that aim to solve presupposed problems of uncertainty-based AL; namely, that it is unscalable and that it is prone to selecting outliers. Furthermore, we explore the influence of the query-pool size on the performance of AL. Whereas it was found that the proposed heuristics for AL did not improve performance of AL; our results show that using uncertainty-based AL with BERT base outperforms random sampling of data. This difference in performance can decrease as the query-pool size gets larger

The OCareCloudS project: toward organizing care through trusted cloud services

The increasing elderly population and the shift from acute to chronic illness makes it difficult to care for people in hospitals and rest homes. Moreover, elderly people, if given a choice, want to stay at home as long as possible. In this article, the methodologies to develop a cloud-based semantic system, offering valuable information and knowledge-based services, are presented. The information and services are related to the different personal living hemispheres of the patient, namely the daily care-related needs, the social needs and the daily life assistance. Ontologies are used to facilitate the integration, analysis, aggregation and efficient use of all the available data in the cloud. By using an interdisciplinary research approach, where user researchers, (ontology) engineers, researchers and domain stakeholders are at the forefront, a platform can be developed of great added value for the patients that want to grow old in their own home and for their caregivers.peerreview_statement: The publishing and review policy for this title is described in its Aims & Scope. aims_and_scope_url: http://www.tandfonline.com/action/journalInformation?show=aimsScope&journalCode=imif20status: publishe